Three-dimensional (3D) television has been a technology trend in recent years that intends to bring viewers sensational viewing experience. Various technologies have been developed to enable 3D viewing and the multi-view video is a key technology for 3DTV application among others. Since all cameras capture the same scene from different viewpoints, multi-view video data contains a large amount of inter-view redundancy. To exploit the inter-view redundancy, additional tools such as advanced residual prediction (ARP) have been integrated to conventional 3D-HEVC (High Efficiency Video Coding) or 3D-AVC (Advanced Video Coding) codec. When ARP is used, the residual for the current block is predicted by a corresponding residual.
There are two types of ARP representing inter-view ARP and temporal ARP. If the current motion vector (MV) is in the inter-view direction (i.e., a disparity MV), the ARP is called inter-view ARP. For the inter-view ARP, the current inter-view residual is predicted by a corresponding inter-view residual from the temporal direction. If the current MV is in the temporal direction, the ARP is called temporal ARP. For the temporal ARP, the current temporal residual is predicted by a corresponding temporal residual from the inter-view direction. The basic concept of the inter-view ARP in existing 3DV-HEVC is illustrated in FIG. 1. The current picture (110) and temporal reference picture (120) correspond to two pictures in the current view. The inter-view prediction residual for a current picture is predicted by the inter-view prediction residual in a different access unit (i.e., different picture order count, POC). For example, an inter-view predictor for the current block (112) in the current picture (110) can be determined based on a motion vector (130) in the inter-view direction (i.e., a disparity vector, DV). The block structure used for the ARP process may correspond to a prediction unit, PU. In other words, the current block corresponds to a current PU. A corresponding temporal reference block (122) for the current block (112) can be located from the current block (112) in the temporal reference picture (120) using a derived MV (DMV 140). The MV (130) in the inter-view direction is also applied to find an inter-view predictor for the temporal reference block (122). The inter-view residual associated with the temporal reference block (122) in a POC different from the POC of the current picture is used to predict the inter-view residual associated with the current block (112).
FIG. 2 illustrates an example of the derivation procedure of DMV for the case in FIG. 1. First, the inter-view reference block (212) in a corresponding inter-view reference picture (210) is located based on the current block (112) using the current MV (130) in the inter-view direction. Since the inter-view reference picture (210) has been coded before the current picture, a motion vector (230) can be determined for the inter-view reference block (212). The MV (230) points to a temporal reference block (222) of a temporal reference picture (220) in the reference view. The MV (230) is scaled to obtain the derived MV (DMV 140). The example shown in FIG. 2 illustrates a case where the MV (230) points to a temporal reference picture (220) that is, based on the display order, temporally farther away from the current picture (110) than the current temporal picture (120). The temporal picture (120) is determined as ARP reference picture in slice level. Therefore, the MV (230) has to be scaled down to derive the DMV (140) in this case. However, the temporal reference picture (220) could also be closer to the current picture (110) than the temporal picture (120). In this case, the MV (230) has to be scaled up to derive the DMV (140). If the POC of the temporal reference picture (220) (i.e., POC1) is equal to the POC of the temporal picture (120) (i.e., POC2), then the scaling has no need.
After the DMV (140) is derived, the temporal reference block (122) in the current view (i.e., dependent view) can be located. The inter-view prediction residual for the temporal reference block (122) can be determined based on the difference between temporal reference block (122) in the dependent view and its corresponding inter-view reference block in the reference view, where the inter-view reference block is located based on the location of the temporal reference block (122) and the MV (130) in the inter-view direction. According to inter-view ARP, the inter-view prediction residual is then used as a prediction for the current inter-view prediction residual.
The scaling process is well-known in the field of video coding. The MV scaling is performed according to the POC distance. The particular implementation adopted in the existing 3D-HEVC is described as follows, where input is the MV (230) and output is the DMV(140):tx=(16384+(Abs(td)>>1))/td  (1)distScaleFactor=Clip3(−4096,4095,(tb*tx+32)>>6)  (2)DMV=Clip3(−32768,32767,Sign(distScaleFactor*MV)*((Abs(distScaleFactor*MV)+127)>>8)),  (3)where td and tb are derived as:td=Clip3(−128,127,POC0−POC1)  (4)tb=Clip3(−128,127,POC0−POC2)  (5)
As shown in equation (1), a division by td operation is performed and td corresponds to the POC distance between POC0 and POC1 (i.e., POC0−POC1), The difference is further clipped to a range from −128 to +127 as shown in equation (4). When POC1 is the same as the POC0, td will be 0. The scaling process becomes meaningless and would result in coder crash since the denominator td is equal to 0 in equation (1). This situation may occur when the MV of the inter-view reference block is in the inter-view direction as shown in FIG. 3. The current MV (330) in the inter-view direction points to an inter-view reference block (312) in an inter-view reference picture (310) in reference view j. However, the MV (340) associated with the inter-view reference block (312) points to another inter-view reference picture (320) the inter-view direction. The POC of said another inter-view reference picture (320) is the same as the POC of the current picture (i.e., POC1=POC0). Therefore, td=POC0−POC1=0 and this will cause an issue in the MV scaling process. It is noted that FIG. 2 and FIG. 3 illustrate DMV derivation process for two different cases. In FIG. 2, the inter-view reference block (212) points to a temporal reference picture. In FIG. 3, the inter-view reference block (312) points to another inter-view reference picture. After DMV is derived, the DMV is used to locate a temporal reference block (122) for the current block as shown in FIG. 1. Furthermore, the inter-view residual predictor is formed and is used to predict the current inter-view residual using the structure shown in FIG. 1.
Therefore, it is desirable to develop a method to overcome this MV scaling issue.